Method of playing chord inversions on a virtual instrument

ABSTRACT

A user interface implemented on a touch-sensitive display for a virtual musical instrument comprising a plurality of chord touch regions configured in a predetermined sequence, each chord touch region corresponding to a chord in a musical key and being divided into a plurality of separate touch zones, the plurality of chord touch regions defining a predetermined set of chords, where each of the plurality of separate touch zones in each region is associated with one or more preselected MIDI files stored in a computer-readable medium. In some embodiments, the touch zones are configured to provide different harmonic configurations of a base chord associated with the chord touch region. Some harmonic configurations provide progressively wider harmonic ranges across each adjacent touch zone. Other harmonic configurations can provide chords with a progressively higher relative pitch across each adjacent touch zone.

CROSS-REFERENCE TO RELATED APPLICATIONS

This is a continuation of U.S. application Ser. No. 13/787,734, filedMar. 6, 2013, which claims benefit under 35 U.S.C. §119 of U.S.Provisional Patent Application No. 61/607,585, filed on Mar. 6, 2012,and entitled “DETERMINING THE CHARACTERISTIC OF A PLAYED NOTE ON AVIRTUAL INSTRUMENT,” which are herein incorporated by reference in theirentirety for all purposes.

BACKGROUND

Virtual musical instruments, such as MIDI-based or software-basedkeyboards, guitars, basses, and the like, typically have user interfacesthat closely resemble the actual instrument. For example, a virtualpiano will have an interface configured as a touch-sensitiverepresentation of a keyboard, or a virtual guitar will have an interfaceconfigured as a touch-sensitive fret board. While these types ofinterfaces may be intuitive, they require that the user understands howto play notes, chords, chord progressions, etc., on a real musicalinstrument in order to implement them on the virtual musical instrument,such that the user is able to produce pleasing melodic or harmonicsounds from the virtual instrument. Such requirements may be problematicfor musicians with little to no experience with the particularcorresponding instrument or in music composition in general.Furthermore, some instruments may not lend themselves to virtualreplicas requiring similar input articulations on the user interface.For example, brass or orchestral instruments may require air pressure orhandheld instruments in order to play a note, much less a chord. MIDIkeyboards are typically used as samplers to recreate these orchestralsounds and, as discussed above, require some level of keyboardproficiency and musical knowledge to produce pleasing melodic orharmonic phrases.

Not all users who would enjoy playing a virtual instrument are musicianswho know how to form chords, construct chord progressions, or composediatonic harmony within a musical key. Furthermore, users who do knowhow to form chords and play chord progressions on a real instrument mayfind it difficult to recreate music with the same musical proficiency onthe user interface due to the lack of tactile stimulus (e.g., weightedkeys, strings, bow, etc.), which the user may be accustomed to.

These problems lead to frustration and make a system less useful, lessenjoyable, and less popular. Therefore, a need exists for a system thatstrikes a balance between simulating traditional musical instruments andproviding an optimized user interface that allows effective musicalinput and performance, and that allows even non-musicians to experiencea musical performance on a virtual instrument.

SUMMARY

Various embodiments provide systems, methods, and devices for musicalperformance and/or musical input that solve or mitigate many of theproblems of existing electronic systems. A user interface presents anumber of chord touch regions, each corresponding to a chord of aparticular key, such as a major or minor key. The chord touch regionsare arranged in a predetermined sequence, such as by fifths within theparticular key. Each chord region includes a number of touch zonesconfigured to detect and distinguish between a number of touch gesturalarticulations including at least one of a legato articulation, apizzicato articulation, or a staccato articulation on a touch zone ofthe user interface. These gestural articulation may simulate varioustechniques used in playing real orchestral stringed instruments,including violins, violas, cellos, basses and the like. Depending on thetouch gesture articulation that is played, a preselected audio file isassociated with the touch zone.

In further embodiments, the touch zones are configured to providedifferent harmonic configurations of a base chord associated with thechord touch region. Some harmonic configurations provide progressivelywider harmonic ranges across each adjacent touch zone. Other harmonicconfigurations can provide chords with a progressively higher relativepitch across each adjacent touch zone.

In yet further embodiments, a legato swipe gesture can trigger playbackto two preselected audio signals for a virtual instrument, where avelocity of the swipe gesture can control characteristics of across-fade and amplitude of the composite audio signal. The twopreselected audio signals can be loud and soft audio signals resemblingthe sound of a bowed stringed instrument having, where the loud audiosignal includes a strong attack and heavy timbre and the soft audiosignal includes a soft attack and light timbre. In some cases, as thevelocity of the swipe gesture increases, the playback of the twopreselected audio signals cross-fade from the soft audio signal to theloud audio signal. In contrast, as the velocity of the swipe gesturedecreases, the playback of the two preselected audio signals cross-fadefrom the loud audio signal to the soft audio signal.

In certain embodiments, a method includes displaying a plurality ofvirtual instruments on a scrollable area of a touch-sensitive userinterface. In response to a detected swipe gesture on the scrollablearea, the virtual instruments cycle through or move across the displayat a speed proportional to the swipe. A virtual instrument can beselected for the user interface based on its position on the scrollablearea.

In some embodiments, a system and method is provided to modulate a lifecycle of a note and, through signal processing, create smoothtransitions between notes and chords played by a user by modeling avirtual momentum created by a user swiping gesture on the interface andmaintaining that momentum throughout subsequent harmonic or melodicprogressions.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a schematic illustration of a user interface for a virtualmusical instrument, according to an embodiment of the invention.

FIG. 2 illustrates a plurality of gestural legato articulations over aplurality of musically defined touch regions, according to an embodimentof the invention.

FIG. 3 illustrates a two-finger auto-legato gestural articulation over aplurality of musically defined touch regions, according to an embodimentof the invention.

FIG. 4 illustrates a pizzicato gestural articulation in a touch regionof a touch-sensitive musical interface, according to an embodiment ofthe invention.

FIG. 5 illustrates a staccato gesture articulation in a touch region ofa touch-sensitive musical interface, according to an embodiment of theinvention.

FIG. 6 is a simplified flow diagram illustrating aspects of a method ofdetermining a characteristic of a note outputted by a virtualinstrument, according to an embodiment of the invention.

FIG. 7 shows an illustration of a user interface for a virtual musicalinstrument, according to an embodiment of the invention.

FIG. 8 illustrates a legato gestural articulation on a touch region of avirtual musical instrument interface, according to an embodiment of theinvention.

FIG. 9 depicts a graph illustrating the cross-fading of two samples of avirtual instrument, according to an embodiment of the invention.

FIG. 10 illustrates a stereo field mapping configuration of across-faded composite sample, according to an embodiment of theinvention.

FIG. 11 depicts a dual flywheel mass modeling system, according to anembodiment of the invention.

FIG. 12 illustrates an embodiment of a note lifecycle, according to anembodiment of the invention.

FIG. 13 is a simplified state machine illustrating aspects of modeling alifecycle of a note played on a virtual instrument, according to anembodiment of the invention.

FIG. 14 illustrates an example of a musical performance system that canenable a user to compose and create music with a number of virtualinstruments on a music application, according to an embodiment of theinvention.

FIG. 15 illustrates a computer system according to an embodiment of thepresent invention.

FIG. 16 illustrates a chord touch region including four touch zones withchord configurations having successively wider harmonic ranges,according to an embodiment of the invention.

FIG. 17 illustrates a chord touch region including four touch zones withchord configurations having harmonic ranges successively higher inpitch, according to an embodiment of the invention.

FIG. 18 illustrates a chord touch region including four touch zones withchord configurations that change by a minimum number of notes betweenchord inversions, according to an embodiment of the invention.

FIG. 19 illustrates a chord touch region including four touch zones withchord configurations that change by a minimum number of notes betweenchords, according to an embodiment of the invention.

FIG. 20 is a simplified flow diagram illustrating aspects of a method ofselecting a virtual instrument on a user interface, according to anembodiment of the invention.

FIG. 21 is a simplified flow diagram illustrating aspects of a method ofmodulating a lifecycle of a musical note or chord, according to anembodiment of the invention.

DETAILED DESCRIPTION

The following disclosure describes systems, methods, and products formusical performance and/or input. Various embodiments can include orcommunicatively couple with a wireless touch screen device. A wirelesstouch screen device including a processor can implements the methods ofvarious embodiments. Many other examples and other characteristics willbecome apparent from the following description.

A musical performance system can accept user inputs and audibly soundone or more tones. User inputs can be accepted via a user interface. Amusical performance system, therefore, bears similarities to a musicalinstrument. However, unlike most musical instruments, a musicalperformance system is not limited to one set of tones. For example, aclassical violin or a classical cello can sound only one set of tones,because a musician's interaction with the physical characteristics ofthe instrument produces the tones. On the other hand, a musicalperformance system can allow a user to modify one or more tones in a setof tones or to switch between multiple sets of tones. A musicalperformance system can allow a user to modify one or more tones in a setof tones by employing one or more effects units. A musical performancesystem can allow a user to switch between multiple sets of tones. Eachset of tones can be associated with a patch (e.g., channel strip (CST)file).

FIG. 14 illustrates an example of a musical performance system that canenable a user to compose and create music with a number of virtualinstruments on a music application, according to an embodiment of theinvention. Musical performance system 1400 can include multiplesubsystems such as a display 1405, one or more processing units 1410,and a storage subsystem 1415. One or more communication paths can beprovided to enable one or more of the subsystems to communicate with andexchange data with one another. The various subsystems in FIG. 14 can beimplemented in software, in hardware, or combinations thereof. In someembodiments, the software can be stored on a transitory ornon-transitory computer readable storage medium and can be executed byone or more processing units.

It should be appreciated that musical performance system 1400 as shownin FIG. 14 can include more or fewer components than those shown in FIG.14, can combine two or more components, or can have a differentconfiguration or arrangement of components. In some embodiments, musicalperformance system 1400 can be a part of a portable computing device,such as a tablet computer, a mobile telephone, a smart phone, a desktopcomputer, a laptop computer, a kiosk, etc.

Display 1405 in some embodiments can provide an interface that allows auser to interact with musical performance system 1400. Display 1405 canbe a monitor or a screen in some embodiments. Through the interface, theuser can view and interact with a GUI 1420 of a musical performancesystem 1400. In some embodiments, display 1405 can include atouch-sensitive interface (also sometimes referred to as a touch screen)that can both display information to the user and receive inputs fromthe user. Processing unit(s) 1410 can include one or more processorsthat each have one or more cores. In some embodiments, processingunit(s) 1410 can execute instructions stored in storage subsystem 1415.System 1400 may also include other types of user input and outputmechanisms such as allowing a user to provide an input based on receivedaccelerometer or gyroscope sensor readings (internal to system 1400) orprovide output such as haptic output based on a desired musicalcharacteristic.

Storage subsystem 1415 can include various memory units such as a systemmemory 1430, a read-only memory (ROM) 1440, and a permanent storagedevice 1450. The system memory can be a read-and-write memory device ora volatile read-and-write memory, such as dynamic random access memory.The system memory can store some or all of the instructions and datathat the processor needs at runtime. The ROM can store static data andinstructions that are needed by processing unit(s) 1410 and othermodules of system 1400. The permanent storage device can be aread-and-write memory device. Some embodiments of the invention can usea mass-storage device (such as a magnetic or optical disk or flashmemory) as a permanent storage device. Other embodiments can use aremovable storage device (e.g., a floppy disk, a flash drive) as apermanent storage device.

Storage subsystem 1415 can store a touch gesture library 1415 thatincludes a number of system recognizable touch gestures 1432 on the GUI1420, MIDI-controlled audio samples 1434 for storing data relating tomusic played on the virtual instruments, and virtual instrument data1436 for storing information about each virtual instrument. Furtherdetail regarding system architecture and auxiliary components thereofare not discussed in detail so as not to obfuscate the focus on theinvention and would be understood by those of ordinary skill in the art.

Chord View Interface

FIG. 1 shows a schematic illustration of a user interface 100 for avirtual musical instrument, according to an embodiment of the invention.FIG. 1 shows the user interface displayed on a tablet computer such asthe Apple iPad®; however the interface could be used on any touch screenor touch-sensitive computing device (e.g., smart phone, personal digitalassistant, and the like). The interface 100 includes channel strip file(CST) sound browser 110, a virtual instrument selector (VIS) 120, anautoplay selector 130, a chord player region 140, and a chord/notesswitch 170. Alternatively, other sound files, or “patches” can be usedinstead of, or in addition to, the CST files, which can be accessed withthe sound browser 110.

The CST sound browser 110 can be configured to select a suite of virtualinstruments, where each suite or ensemble is suited for a differentmusical preference. Selecting a given suite (e.g., Symphonic Strings,String Quartet, Baroque Strings, etc.) loads the appropriate CST andautoplay grooves specifically suited to the selected ensemble. Forexample, the Symphonic CST can include a first violin, a second violin,a viola, a cello, and a bass. In contrast, the String Quartet CST mayinclude a first a second violin, a viola, and cello. Any number andconfiguration of virtual instruments can be grouped together in anypreferred combination. In some embodiments, MIDI sequences can be usedinstead of, or in addition to, the autoplay grooves.

The interface 100 includes a number of chord touch regions (chordstrips) 150, shown for example as a set of eight adjacent columns orstrips. Each touch region 150 can correspond to a pre-defined chord(base chord) within one or more particular keys, with adjacent regionsconfigured to correspond to different chords and progressions within thekey or keys. For example, the key of C major includes the diatonicchords of C major (I), D minor (ii), E minor (iii), F major (IV), Gmajor (V), A minor (vi), and B diminished (vii), otherwise known as theTonic, Supertonic, Mediant, Subdominant, Dominant, Submediant, andLeading Tone chords. In the example shown in FIG. 1, an additionalnon-diatonic chord of B-flat major is included with the key of C major,and the chords are arranged sequentially in accordance with the circleof fifths. This arrangement allows a user to easily create sonicallypleasing sequences by exploring adjacent touch regions. A user can touchor initiate gestures (e.g., swipe gestures) on the various touch regions150 to create sounds that are related to the chord that is assigned tothe particular touch region. The types of touches, sounds, and/orgestures associated with the touch regions and features therein arefurther described below. It should be noted that any set of pitches(collectively known as a chord) can be defined for any chord touchregion and is not limited to the conventional harmony described in thisparticular example.

Each chord touch region is divided into a number of touch zones referredto collectively as touch zones 160. Referring to FIG. 1, each touchregion 150 comprises four individual touch zones 162, 164, 166, 168.Each of the touch zones 160 can correspond to various harmonic voicings(e.g., chord inversions) of the base chord assigned to the touch region150 that it belongs to. The chords assigned to each touch zone 160 canbe associated with MIDI-controlled audio samples (e.g., signals,streams, etc.). Touching or initiating touch gestures on any touch zone160 in a region 150 plays the chord audio sample assigned to that touchzone.

MIDI (Musical Instrument Digital Interface) is an industry-standardprotocol defined in 1982 that enables electronic musical instruments,such as keyboard controllers, computers, and other electronic equipment,to communicate, control, and synchronize with each other. For example,MIDI can be used to trigger playback of an audio sample to create asound. In other words, MIDI is an instruction communications protocoltypically used in electronic musical instruments. It should be notedthat the term “audio signal” can also include a MIDI-controlled audiosample or digital signal processor (DSP) generated audio stream. Otheraudio processing systems and protocols may be used as would be known byone of ordinary skill in the art.

Virtual Instrument Selection in Chords View

A user can select a virtual instrument via the VIS 120. In some cases,the virtual instruments can include one or more violins, violas, cellos,basses, or the like. When a user selects one or more instruments withthe VIS 120, the system loads the appropriate patch files (e.g., CSTfiles) for the selected instruments. In some embodiments, the VIS 120displays up to 5 instruments or more, which when selected (e.g., bytapping) will either activate or deactivate a note being played by thatparticular instrument. For example, while performing gestural inputs onthe touch zone region corresponding to a given chord sequence, a usermay select a violin and a bass to play (i.e., output an audible musicaloutput) the given chord sequence. During the input of gestures to createthe chord sequence, the user can deselect the bass in real-time, leavingonly the violin sample to be voiced during the chords sequence.Likewise, reselecting the bass thereafter will reintroduce the basssample's voice into the chord sequence.

Grooves/Auto-Play Selection

In certain embodiments, the interface 110 includes various chordsequence auto-play features. An auto-play selector 130 is configured toloop one or more MIDI-controlled audio samples (e.g., audiosignals/streams, audio loops, etc.) that include a number of predefinedaccompanying rhythms when selected by a user. In response to a userselection, a groove plays for the chord being touched by the user. Insome embodiments, the groove rhythm latches or stops when the usertouches the same chord region again. The groove rhythm can switch to anew chord by selecting a different touch zone 150. The playback tempocan be locked during playback or manually adjustable as required.Variations of the auto-play grooves can be selected by one or more tapson the auto-play selector 130. In some cases, the auto-play grooves canbe a MIDI sequence. Furthermore, the auto-play grooves (or MIDIsequences) will also correspond to the chord voicing selected for eachtouch zone 160.

Chord Inversions

In certain embodiments, each of the touch zones 160 of a touch region150 can include various inversions of the base chord. For example, thetop most touch zone 162 can correspond to a root position of the basechord, such as a C Major triad C-E-G with C in the bass. The nextadjacent touch zone 164 can correspond to a first inversion of the basechord, or the C Major triad C-E-G with E in the bass. The followingadjacent touch zone 166 can correspond to a second inversion of the basechord, or the C Major triad C-E-G with G in the bass. The next adjacenttouch zone 168 can correspond to a third inversion of the base chord, orthe C Major triad C-E-G with an additional 7^(th) in the bass (e.g.,Cmaj7 with B in the bass or a C7 with a Bb in the bass). Alternatively,the touch zone 168 may include a 10^(th) in the bass, or E-C-G, wherethe E note is one octave higher than the E note in the first inversionE. Optional adaptations of the first, second, and third inversions canbe utilized.

In some embodiments, each of the touch zones 160 of a touch region 150can include various inversions of the base chord that are successivelyhigher in relative pitch than the base chord. To illustrate, the topmost touch zone 162 may correspond to a root position of the base chord,such as a C Major triad C-E-G with C in the bass. The next adjacenttouch zone 164 may correspond to a first inversion of the base chord, orthe C Major triad C-E-G with E in the bass, where at least one note ofthe chord (e.g., the top note) is higher than any note of the basechord. Similarly, the following touch zone 166 may include a secondinversion of the base chord, where at least one note of the secondinversion chord (e.g., the top note) is higher in pitch than any note ofthe base chord or first inversions. Finally, the last adjacent touchzone 168 can include a third inversion of the base chord, where at leastone note of the third inversion chord is higher in pitch than any noteof the base chord or first inversions. Configuring inversions in thismanner can make cycling through the touch zones 160 sounds as if therelative pitch is moving from one register to a higher register.

In further embodiments, each of the touch zones 160 of a touch region150 can include various chord voicings (i.e., the harmonic arrangementof notes) that have successively wider harmonic ranges. To illustrate,the top most touch zone 162 may correspond to the base chord with acertain harmonic range. For example, touch zone 162 can include a Ctriad comprising C-E-G where the G can be 3.5 whole steps from the rootC. The next touch zone 162 may correspond to a chord voicing (e.g.,inversion) with a wider harmonic range than the base chord of touch zone162. For example, touch zone 162 can include a C triad comprising C-E-Gwith E in the bass and a C as the top note, where the top note C is 4steps from the root E. The third and fourth touch zones 166 and 168 mayfollow in kind.

FIG. 16 illustrates a chord touch region 1600 including four touch zoneswith chord configurations having successively wider harmonic ranges,according to an embodiment of the invention. The chord touch region 1600is assigned base chord C major and includes touch zones 1610, 1620,1630, and 1640. In some embodiments, each touch zone has a differentchord voicing of the base chord. As described above, the term “chordvoicing” relates to a harmonic arrangement of notes. For example, a Cmajor chord has the notes C-E-G. One chord configuration can have a Cnote as the bass note, while a second configuration may have the G noteas the bass note. Each of these configurations can be referred to as achord voicing. The distance between notes can be measured by half stepsand whole steps, which can be thought of as musical units ofmeasurement. Touch zone 1610 has a first voicing (i.e., harmonicarrangement) 1615 of a C major chord. The harmonic range of the firstvoicing is 3.5 whole steps. Touch zone 1620 has a second voicing 1625 ofthe C major chord with a wider harmonic range than the first voicingspanning 4 whole steps. Touch zone 1630 has a third voicing 1635 of theC major chord with a wider harmonic range than the second voicingspanning 8 whole steps. Touch zone 1640 has a fourth voicing 1645 of theC major chord with a wider harmonic range than the third voicingspanning 10 whole steps. Different chords, chord configurations,inversions, key signatures, and the like, can be customized topreference.

FIG. 17 illustrates a chord touch region 1700 including four touch zoneswith chord configurations having harmonic ranges successively higher inpitch, according to an embodiment of the invention. The chord touchregion 1700 is assigned base chord C major and includes touch zones1710, 1720, 1730, and 1740. In some embodiments, each touch zone has adifferent chord voicing of the base chord. As described above, the term“chord voicing” relates to a harmonic arrangement of notes. Touch zone1710 has a first voicing (i.e., harmonic arrangement) 1715 of a C majorchord. Touch zone 1720 has a second voicing 1725 of the C major chordwith a higher pitch than the first voicing. In other words, both the Eand C notes in chord voicing (“chord”) 1725 are higher in pitch than thehighest note (G) of chord 1715. In some cases, the audible effect ofonly one note in a first chord being higher in pitch than all notes of asecond chord will make the first chord sound higher in pitch overall.Referring back to FIG. 17, touch zone 1730 has a third voicing 1735 ofthe C major chord with a higher pitch than the first (1715) and secondvoicings (1725). For example, the G note is higher in pitch than any ofthe notes of chord 1715 and 1725. Touch zone 1740 has a fourth voicing1745 of the C major chord with a higher pitch than the first (1715),second voicings (1725), and third voicings (1735). For example, the Cnote is higher in pitch than any of the notes of chords (i.e., voicings)1715, 1725, and 1735. It should be noted that other chords, chordconfigurations, inversions, key signatures, and the like, can becustomized to preference.

In some embodiments, touching or articulating gestures up or downthrough the touch zones 160 may cause the chord voicing to change by theminimum number of notes needed to switch to the nearest inversion fromthe chord voicing that was being played prior to the touch or gesturearticulation. For example, if touch zone 162 includes a C major triadC-E-G with C in the bass, and touch zone 164 includes a C major triadE-G-C with an E in the bass, the E and G notes will not change betweenchords. Changing the minimum amount on notes in a chord change canresult in a smoother or more harmonically pleasing sound, as furtherillustrated in FIG. 18.

FIG. 18 illustrates a chord touch region including four touch zones withchord configurations that change by a minimum number of notes betweenchord inversions, according to an embodiment of the invention. The chordtouch region 1800 is assigned base chord C major and includes touchzones 1810, 1820, 1830, and 1840. Touch zone 1810 includes a C majortriad C-E-G with C in the bass, and touch zone 1820 includes a C majortriad E-G-C with an E in the bass. It should be noted that the E and Gnotes do not change during the chord change 1815. As described above,changes between chord inversions (e.g., 1810 to 1820) result in a changein the minimum amount of notes needed for the chord change. In somecases, this can be a dynamic process such that different chord voicingsmay be assigned to the same touch zone depending on the arrangement ofthe previous chord voicing. For example, the C chord inversion assignedto 1810 may produce (e.g., play back) a chord voicing of a differentarrangement when approached from touch zone 1820 rather than from touchzone 1830 because the arrangement of the notes and the minimum changesrequired for the chord change is different. A first inversion C chordwith an E in the bass may only require one note to shift positions whenswitching to a C chord with a C root note (e.g., C note moves to thebass position and other notes do not change). However, a secondinversion chord with a G in the bass may require multiple notes to shiftpositions when switching to a C chord with a C root note. It should benoted that different chords, chord configurations, inversions, keysignatures, and the like, can be customized to preference.

In other embodiments, touching or articulating gestures between chordregions may cause chord voicings to change by the minimum number ofnotes needed to switch between chords. For example, if one touch regionincludes a C major triad C-E-G as a base chord, and a second touchregion includes an E minor triad E-G-B, the E and G notes does notchange between the chord change. Changing the minimum number of notesbetween chord changes can result in more harmonically pleasing chordprogressions.

FIG. 19 illustrates a chord touch region including four touch zones withchord configurations that change by a minimum number of notes betweenchords, according to an embodiment of the invention. The chord touchregion 1900 is assigned base chord C major and includes touch zones1910, 1920, 1930, and 1940. The chord touch region 1950 is assigned basechord E minor and includes touch zones 1960, 1970, 1980, and 1990. Touchzone 1910 includes a C major triad C-E-G with C in the bass, and touchzone 1960 includes a E minor triad E-G-B with an E in the bass. Asdescribed above, changes between chords result in a change in theminimum amount of notes needed for the chord change. In this case, the Eand G notes do not change position during the chord change 1935. In somecases, this can be a dynamic process such that different chord voicingsmay be assigned to the same touch zone depending on the arrangement ofthe previous chord voicing. For example, the E minor chord inversionassigned to 1970 may produce (e.g., play back) a chord voicing of adifferent when approached from touch zone 1910 rather than from touchzone 1920 because the arrangement of the notes and the minimum changesrequired for the chord change may be different. A described above,changing from a C chord with a C note in the bass to an E minor chordwith an E in the bass results in only one note change (i.e., C changesto B), while both the E and G notes do not change. However, changingfrom a second inversion C chord (1930) with a G in the bass to an Emchord with E in the bass position may require multiple notes to shiftposition. It should be noted that different chords, chordconfigurations, inversions, key signatures, and the like, can becustomized to preference.

Touch Gesture Articulations in Chord View

Each touch zone 160 in each touch region 150 is configured to detect oneor more of a number of different touch gestural articulations, accordingto certain embodiments. The touch gesture articulations can include alegato articulation, a pizzicato articulation, and a staccatoarticulation. Each of the gesture articulations triggers playback (i.e.,output) of a corresponding audio signal that simulates the selectedarticulation. In some cases, the audio signal can be a MIDI-controlledaudio sample or a DSP generated audio stream. It should be noted thatthe term “audio signal” can also any suitable means of playing an audiosignal. Playing an audio signal can include playback on anelectro-acoustic transducer (e.g., a speaker), decoding the audiosignal, saving in memory, coupling to other systems or devices, or othernon-audible output scenarios that would be appreciated by one ofordinary skill in the art with the benefit of this disclosure. It shouldbe noted that although three different gesture articulations aredescribed, any number and/or type of gesture articulations (e.g., doubletaps, dual finger swipes, palm, circular motions, polygonal outlinemotions, multi-finger gestures, etc.) may be used in any combination orset as required.

FIG. 2 illustrates a plurality of legato articulations over a pluralityof touch regions 250, according to an embodiment of the invention. Thetouch regions 250 and touch zones 260 are similar to those which aredescribed in FIG. 1. The legato articulation is initiated by atap-to-swipe gesture on one of the touch zones 260. In certain cases,the initial tap should be longer than a predetermined period of timebefore initiating the swipe gesture to differentiate the legatoarticulation from the staccato (i.e., tap gesture shorter than thepredetermined time) or pizzicato articulation (i.e., swipe gesture withno initial tap period) further described below. A continuous up-and-downswiping motion of the legato articulation mimics the motion of a bowplayed across a set of strings on a stringed instrument (e.g., violin,cello, bass). Similarly, the corresponding audio signal (e.g.,MIDI-controlled audio sample) plays the sound of a bowing action acrossa set of strings on a stringed instrument. In FIG. 2, gesture 270illustrates a touch-and-swipe legato gesture on touch zone 264A. Theinitial touch point on the touch zone 264A determines the inversion ofthe base chord that is played. In this example, the legato articulation270 plays a first inversion legato bowing sound in A minor. Gesture 280illustrates a touch-and-continuous swipe legato gesture at touch zone264B. In this example, the legato articulation 280 plays a firstinversion legato bowing sound in C major and continues to do so even ifthe swiping motion moves outside of the associated touch region. In someembodiments, as long as the finger is not removed from the touch surfaceof the interface 100, the bowing action will continue to play the sameinversion of the initial touch point regardless of the subsequentposition of the finger on the interface. In some cases, changing basechords or chord inversions occurs by starting a new legato gesturearticulation in a different zone by lifting the finger and placing it inthe desired chord/inversion region. Alternatively, introducing a secondfinger in a different touch zone can begin a new legato gesturearticulation. In other words, the interface recognizes the second touchinput as a new legato gestural articulation and ignores the firstfinger. Furthermore, the speed or velocity of the legato articulationcan determine various expression values of the corresponding audiosignal, such as a volume of playback. Other expression values caninclude pitch, duration, equalization, filtering (e.g., high pass, lowpass, band pass, notch filters, etc.), pan, effects (e.g., reverb,chorus, delay, distortion, presence, etc.), or any other parameters thatcan be modulated as described. In some embodiments, the direction and/oracceleration of the gesture articulation can also be used to modulateone or more expression values (e.g., variable of playback).

In certain embodiments, multi-finger gestures can allow the user toquickly transition from one chord zone to another. For example, if asecond touch point is detected within the chord interface while a bowinggesture is active in another chord touch zone, the chord can quicklychange to the new chord touch zone. Furthermore, if the second touch isa bow swipe (e.g., legato articulation), the expression (e.g., a volume)value generated from the first touch transfers to the new expressionvalue for the second touch using the same swipe gesture ballistics thatwere present immediately prior to the change. To illustrate, if twoconsecutive swipes are occurring at substantially the same swipe gesturespeed, then there should be substantially no change in expression value(volume) when the chord transition occurs. In contrast, if the secondgesture occurs at a different speed than the first gesture, then theexpression value (volume) transitions from the first expression value tothe second expression value in a smooth and musical way, as furtherdescribed below.

FIG. 3 illustrates a two-finger auto-legato gesture articulation,according to an embodiment of the invention. FIG. 3 depicts a firsttouch 370 (tap-and-hold) and simultaneous second touch 372(tap-and-hold) at touch zone 364A of the touch region 350A. Tapping andholding two fingers in a touch zone auto-plays a legato chord sound. Inthis example, the auto legato gesture articulation plays the inversionassociated with touch zone 364A of touch region 350A (A minor). Twofinger auto-legato gestures do not require a swipe gesture and can beconfigured to play instantly. In some cases, the volume of theauto-legato articulation is fixed and can be based on the initialvelocity of the two finger-tap gesture. The initial velocity can bedetermined, for example, by an accelerometer. In certain cases wheremultiple touch zones are simultaneously touched, a resultant touchlocation can be selected as the initial touch zone. For example, a firsttouch 380 at touch zone 364B and a simultaneous second touch 382 attouch zone 368B results in an effective two finger-tap gesture 384 attouch zone 366B of touch region 350B. The note characteristics of thetwo-finger auto-legato touch gestures typically have relatively shortattack times to support rapid transitions to different base chords orinversions.

FIG. 4 illustrates a pizzicato gesture articulation in a touch region450 of a touch-sensitive interface, according to an embodiment of theinvention. The pizzicato gesture articulation can be initiated by aquick tap gesture 480 including an initial tap in one of the pluralityof separate touch zones 460 and a release prior to a predeterminedperiod. The pizzicato gestural articulation triggers the playback of asecond corresponding MIDI-controlled audio sample that simulates thesound of a plucking action on a stringed instrument. In one embodiment,the predetermined period is any suitably short period of time (e.g.,less than 200 ms) that would be appreciated by one of ordinary skill inthe art with the benefit of this disclosure. In some embodiments, if thetouch is held longer than the predetermined period, the removal of thefinger will not trigger the audio sample. Furthermore, the speed orvelocity of the pizzicato gestural articulation (e.g., the speed atwhich the initial tap on the interface is received) can determinevarious expression values of the corresponding audio sample, such as avolume of playback. Other expression values can include pitch, duration,equalization, filtering (e.g., high pass, low pass, band pass, notchfilters, etc.), pan, effects (e.g., reverb, chorus, delay, distortion,presence, etc.), or any other parameters that can be modulated asdescribed. The initial velocity of the initial tap may be determined byan accelerometer.

FIG. 5 illustrates a staccato gesture articulation 580 in a touch region550 of a touch-sensitive interface, according to an embodiment of theinvention. The staccato gestural articulation is initiated by anin-motion swipe gesture beginning upon contact with a touch zone 560. Inother words, a quick swipe motion is initiated with no pause (e.g., tap)before beginning the swipe, where the swipe begins upon contact with thegiven touch zone 560. The staccato gesture articulation triggersplayback of a corresponding MIDI controlled audio signal that simulatesthe sound of a short burst of a bowing action on a stringed instrument.The speed or velocity of the staccato articulation can determine variousexpression values of the corresponding audio sample, such as a volume ofplayback. Other expression values can include pitch, duration,equalization, filtering (e.g., high pass, low pass, band pass, notchfilters, etc.), pan, effects (e.g., reverb, chorus, delay, distortion,presence, etc.), or any other parameters that can be modulated asdescribed.

FIG. 6 is a simplified flow diagram illustrating aspects of a method 600of determining a characteristic of a note played on a virtualinstrument, according to an embodiment of the invention. The method 600is performed by processing logic that may comprise hardware (e.g.,circuitry, dedicate logic, etc.), software (which as is run on a generalpurpose computing system or a dedicated machine), firmware (embeddedsoftware), or any combination thereof. In one embodiment, the method 600is performed by the processor unit 1410 of FIG. 14.

Referring to FIG. 6, the method 600 begins with the receiving a userinput on a touch zone of a touch sensitive interface (610). The userinput can include one of a number of touch gesture articulations. At620, the processor determines if the touch gesture is a legatoarticulation. A legato articulation includes a touch-to-swipe gesturethat imitates the motion of a bow played across a set of strings on astringed instrument (e.g., violin, cello, bass). In certain cases, theinitial touch should be longer than a predetermined period of timebefore initiating the swipe gesture to differentiate the legatoarticulation from the staccato (i.e., tap gesture shorter than thepredetermined time) or pizzicato articulation (i.e., swipe gesture withno initial tap period). If the processor determines that the user inputis a legato articulation, the processor plays (or outputs) one of anumber of preselected audio signals (e.g., MIDI controlled audiosamples, DSP generated audio streams, etc.) on an output device (e.g.,speaker, headphones, or the like) corresponding to both the touch regionselected (e.g., chord inversions) and the legato articulation (650). Ifthe processor determines that the user input is not a legatoarticulation, the method proceeds to 630.

At 630, the processor determines if the touch gesture is a pizzicatoarticulation. A pizzicato gesture articulation can be initiated by aquick tap gesture including an initial tap in a touch zone and a releaseprior to a predetermined period. The pizzicato articulation imitates thesound of a plucking action on a stringed instrument. If the processordetermines that the user input is a pizzicato articulation, theprocessor plays one of a number of preselected audio signals on anoutput device corresponding to both the touch region selected (e.g.,chord inversions) and the pizzicato articulation (650). If the processordetermines that the user input is not a pizzicato articulation, themethod proceeds to 640.

At 640, the processor determines if the touch gesture is a staccatoarticulation. The staccato articulation can be initiated by an in-motionswipe gesture (i.e., no pause or tap) beginning upon contact with atouch zone. The staccato gesture articulation imitates the sound of ashort burst of a bowing action on a stringed instrument. If theprocessor determines that the user input is a staccato articulation, theprocessor plays one of a number of preselected audio signals (e.g.,MIDI-controlled audio samples) on an output device corresponding to boththe touch region selected (e.g., chord inversions) and the staccatoarticulation (650). If the processor determines that the user input isnot a staccato articulation, the method ends.

Differentiation between each of the legato and staccato gesturalarticulations is based on detecting a time interval associated with theinitial touch which is followed by a consecutive (i.e., without a liftoff of the touch) swiping gesture having a particular initial speed. Alegato articulation is detected based on the detecting of an initialtouch lasting a given time interval, which is followed by a swipingmotion of any speed. The staccato gestural articulation is based ondetecting that the initial touch is less than the given time interval.

It should be appreciated that the specific steps illustrated in FIG. 6provides a particular method of determining a characteristic of a noteplayed on a virtual instrument, according to an embodiment of thepresent invention. Other sequences of steps may also be performedaccording to alternative embodiments. In certain embodiments, the method600 may perform the individual steps in a different order, at the sametime, or any other sequence for a particular application. For example,the pizzicato gesture may be determined before the legato gesture.Moreover, the individual steps illustrated in FIG. 6 may includemultiple sub-steps that may be performed in various sequences asappropriate to the individual step. Furthermore, additional steps may beadded or removed depending on the particular applications. One ofordinary skill in the art would recognize and appreciate many variation,modification, and alternatives of the method.

Notes View Interface

FIG. 7 shows an illustration of a user interface 700 for a virtualmusical instrument, according to an embodiment of the invention. FIG. 7shows the user interface displayed on a tablet computer such as theApple iPad®; however the interface could be used on any touch screen ortouch-sensitive computing device (e.g., smart phone, personal digitalassistant, and the like). The interface 700 includes CST sound browser710, a virtual instrument selector (VIS) 720, a violin neck andheadstock 730, an Arco/Pizz badge 740, and a chords/notes switch 770.The violin neck 730 includes a plurality of virtual strings 750, whereeach string corresponds to a base note (open string) and is divided intoa plurality of touch zones 760 with notes corresponding to apredetermined scale (e.g., chromatic, pentatonic, harmonic minor, etc.).Each of the notes can correspond to one or more MIDI-controlled audiosamples associated with the selected virtual instrument, the type ofarticulation performed, expression value, or any combination thereof, asfurther described below.

The chords/notes switch 770 functions as a virtual toggle switchconfigured to switch between the chords view (e.g., FIG. 1) and thenotes view (e.g., FIG. 7). The CST sound browser 710 can be configuredto select a suite of virtual instruments and their associated audiosignals (e.g., MIDI-controlled audio sample, DSP generated audio stream,etc.), where each suite or ensemble is suited for a different musicalpreference. Selecting a given suite (e.g., Symphonic Strings, StringQuartet, Baroque Strings, etc.) loads the appropriate CST filesspecifically suited to the selected ensemble. For example, the SymphonicCST can include a first violin, a second violin, a viola, a cello, and abass. In contrast, the String Quartet CST may include a first a secondviolin, a viola, and cello. Any number of virtual instruments can begrouped together in any preferred configuration.

Virtual Instrument Selection in Notes View

The VIS 720 is used to select the virtual instruments desired by theuser. In some cases, the virtual instruments can include one or moreviolins, violas, cellos, basses, or the like. When a user selects one ormore instruments with the VIS 720, the system will load the appropriateaudio samples for the selected instruments. In some embodiments, oneinstrument can be played at a time in notes view. For example, a usermay select a cello, thereby deactivating any basses, violins, and/orviolas in that particular suite of virtual instruments. As shown in FIG.7, only one virtual instrument is fully shown at one time to clearlyidentify the active virtual instrument. In alternative embodiments,multiple instruments can be selected for simultaneous playback in notesview. For example, the VIS 720 (“part selector”) can display up to 5instruments or more, which when selected (e.g., by tapping) eitheractivates or deactivates a note being played by the selectedinstruments.

In certain embodiments, the VIS 720 displays a number of virtualinstruments (e.g., violins, violas, cellos, and basses) that arearranged on a virtual scrollable area on the user interface 700. A swipegesture on the virtual scrollable area can cause the area to slide in adirection relative to the swipe gesture, thereby changing an orientationof the virtual scrollable area. For example, as a user performs a swipegesture on the VIS 720 from right-to-left, the virtual instruments cyclethrough or moves across the VIS 720 display area from right-to-left at aspeed proportional to the speed of the swipe gesture. Selecting avirtual instrument can be based on the orientation of the virtualscrollable area. In addition, the virtual scrollable area can have aselection location 790 to select the virtual instrument located in theselection location 790. In some cases, the selection location is locatedin the center position of the VIS 720. Referring to FIG. 7, the cello iscurrently selected because it is positioned in the selection location790. It should be noted that the configuration of the VIS 720 can becustomized to a user preference. For example, all virtual instrumentsmay be visible on the VIS 720 or only a subset at any given time. Thevirtual scrollable area may slide linearly, or on a virtual axis of asuitable radius such that the virtual instruments appear to “rotate”into position at the selection location 790. In some embodiments, aselection may be made automatically as a virtual instrument slides intothe selection location 790, or a selection may require an additionalmanual tap on the virtual instrument to confirm selection.

FIG. 20 is a simplified flow diagram illustrating aspects of a method2000 of selecting a virtual instrument on a user interface, according toan embodiment of the invention. The method 2000 is performed byprocessing logic that may comprise hardware (e.g., circuitry, dedicatelogic, etc.), software (which as is run on a general purpose computingsystem or a dedicated machine), firmware (embedded software), or anycombination thereof. In one embodiment, the method 2000 is performed bythe processor unit 1410 of FIG. 14.

Referring to FIG. 20, the method 2000 begins with displaying a pluralityof virtual instruments on a scrollable area of a user interface (2010).In some cases, the user interface can be touch-sensitive, where thescrollable area is directly accessible by user touch (e.g., finger,stylus, etc.). In non-touch sensitive embodiments, the scrollable areamay be accessible via input device (e.g., mouse, keyboard, etc.). At2020, the user interface (by way of a processor device) detects a swipegesture on the virtual scrollable area. In response to the swipegesture, the user interface scrolls the scrollable area (e.g., VIS 720)and virtual instruments (e.g., virtual violin, viola, cello, bass, etc.)thereon in a direction relative to the swipe gesture (2030). In certainembodiments, as a user performs a swipe gesture on the scrollable area(e.g., VIS 720) from right-to-left, the virtual instruments cyclethrough or moves across the VIS 720 display area from right-to-left at aspeed proportional to the speed of the swipe gesture. The user interfacethen selects a virtual instrument based on an orientation of the virtualscrollable area (2040). The virtual scrollable area can have a selectionlocation configured within the scrollable area to identify the virtualinstrument to be selected. For example, if the user interface scrollsthrough the virtual instruments in response to a swipe gesture and aviolin stops in the selection location, then the violin will beselected. In some cases, the selection location is located in the centerposition of the scrollable area. The selection location can optionallybe configured in any suitable position on the scrollable area. At 2050,the user interface assigns the selected virtual instrument to a firsttouch zone on the user interface. The user interface further assign oneor more MIDI-controlled audio samples to the first touch zone where theaudio samples correspond to the selected virtual instrument (2060)

It should be appreciated that the specific steps illustrated in FIG. 20provides a particular method of selecting a virtual instrument on a userinterface, according to an embodiment of the present invention. Othersequences of steps may also be performed according to alternativeembodiments. In certain embodiments, the method 2000 may perform theindividual steps in a different order, at the same time, or any othersequence for a particular application. Moreover, the individual stepsillustrated in FIG. 20 may include multiple sub-steps that may beperformed in various sequences as appropriate to the individual step.Furthermore, additional steps may be added or removed depending on theparticular applications. One of ordinary skill in the art wouldrecognize and appreciate many variation, modification, and alternativesof the method.

Touch Gesture Articulation in Notes View

Each of the virtual strings 750 of a user interface 700 is configured todetect one or more of a plurality of different touch gesturearticulations, according to certain embodiments. The touch gesturearticulations can include a legato articulation, a pizzicatoarticulation, and a staccato articulation. Each of the gesturearticulations triggers playback of a corresponding MIDI-controlled audiosample (or DSP generated audio stream) that simulates the selectedarticulation. As described above, the legato articulation mimics abowing action across a set of strings, the pizzicato articulation mimicsa sound of plucking a stringed instrument, and a staccato articulationmimics a sound of a short burst of a bowing action on a stringedinstrument.

In some embodiments, the Arco|Pizz (i.e., bow/pizzicato) badge 740 isdisplayed at the far left of the notes view inside the headstockgraphic. The arco/pizz badge (“button badge”) 740 is a toggle switchconfigured to select between a first mode (auto-bow mode) and secondmode (arco|pizz mode) of operation. In some embodiments, the buttonbadge 740 is a temporary switch where the badge remains in the secondmode of operation as long as a user is pressing the badge 740. In otherembodiments, the badge 740 toggles between the two modes of operationand remains in the selected mode of operation until the badge 740 isdepressed again.

In the auto-bow mode of operation (i.e., first mode), touching any pointon one of the strings 750 can produce a corresponding legato note with ashort attack. The legato note plays continuously as long as the touchobject (e.g., finger, stylus, or any input device) remains on thestrings. In certain embodiments, the first mode of operation interpretsany gesture (e.g., tap, swipe, etc.) as a touch and plays thecorresponding legato note, regardless of the initiating gesture.

FIG. 8 illustrates a legato articulation on the notes view interface800, according to an embodiment of the invention. When a user pressesthe button badge 840 and holds a point 880 on a string 864, a graphicindicator 885 is shown prompting the user to perform an up and downswiping (bowing) motion. In some cases, the graphic indicator 885 is anote highlight radiating from the initial touch point 880 to indicatethe bow motion gesture to the user. The interface 800 typically displaysthe graphic indicator 885 when the button badge 840 is active, a touchis detected on a string, and the touch is determined not to be apizzicato tap (i.e., the tap is longer than a predetermined period). Asthe user swipes up and down, the instrument plays a legato note based onthe start position of the bowing action. While the badge button 840 isheld, the user can tap any string position for a corresponding pizzicatonote. In certain embodiments, only one instrument can be played at atime (violin, viola, cello, bass) in the notes view. In alternativeembodiments, multiple instruments can be selected in the notes view forsimultaneous play.

With the button badge 840 pressed, both legato and staccatoarticulations can be performed in a similar way as in the chords view.The speed or velocity of the staccato articulation (i.e., how hard thenote is tapped) can determine various expression values of thecorresponding MIDI-controlled audio sample, such as a playback volume. Afaster legato articulation can create a louder volume and slower legatoarticulations may create a softer volume. The notes view can be furtherconfigured to support multi-finger detection (e.g., double stop chords)with staccato, legato, and pizzicato articulations. In certain aspects,the instrument strings in the notes view are tuned to the standard openstrings for the selected virtual instrument. In some embodiments, notesliding or pitch bending can be supported when the button badge 840 isnot active. An accelerometer can optionally be configured to determinethe velocity (i.e., how hard the note is touched) of any articulationsin notes view.

Velocity Sample Cross Fades

According to certain embodiments, the virtual instruments describedherein utilize a cross-fade of two channels comprising a loud sample anda soft sample to create notes of varying velocities (i.e., amplitudes,volume, etc.). Lower velocity audio signals created by the virtualinstruments may have a higher weighted value for the soft sample, andhigher velocity audio signals may have a higher weighted value for theloud sample. The resultant composite sample creates a more realisticsounding virtual instrument that more accurately reproduces the variousaural characteristics and harmonic structures of notes played at varyingdegrees of amplitude.

FIG. 9 depicts a graph 900 illustrating the cross-fading of two samplesof a virtual instrument, according to an embodiment of the invention.Graph 900 includes an envelope level axis 905, a sample output levelaxis 910, a soft sample 920, and a loud sample 930. As described above,a legato swipe gesture can trigger playback of two preselectedMIDI-controlled audio samples for a virtual instrument, where thevelocity of the swipe can control characteristics of the cross-fade andthe amplitude of the composite signal. In some aspects, the soft sample920 can be a first preselected audio sample that includes an audiosample of a bowed stringed instrument having a soft attack and a lighttimbre. The loud sample 930 can be a second preselected audio sampleincluding an audio sample of a bowed stringed instrument having a strongattack and a heavy timbre. As the velocity of the swipe gestureincreases, the playback (i.e., envelope level) of the two preselectedaudio samples cross-fades from the soft sample 920 (i.e., firstpreselected MIDI file) to the loud sample 930 (i.e., second preselectedMIDI file). In contrast, as the velocity of the swipe gesture decreases,the playback of the two preselected audio samples cross-fades from theloud sample 930 to the soft sample 920. In FIG. 9, at an envelope level(e.g., swipe velocity) of 75 out of 128 levels, the loud sample 930cross-fades at a higher output level than the soft signal 920. In someembodiments, the playback volume of the cross-faded composite of the twopreselected audio samples increases in response to an increased velocityof a legato swipe gesture. In contrast, the playback volume of thecross-faded composite of the two preselected audio samples decreases inresponse to a decreased velocity of a legato swipe gesture.

FIG. 10 illustrates a stereo field mapping configuration 1000 of across-faded composite sample, according to an embodiment of theinvention. In each MIDI channel (e.g., virtual instrument), theresulting composite mix of the soft sample and loud sample is summed toa mono signal and panned in a mixer to present a particular stereospread of the five MIDI channel instruments (e.g., the violins, violas,cellos, and basses) in a stereo field. In other cases, a stereo signalis placed within the stereo field for the final output. In thisparticular example, the 1^(st) violin, 2^(nd) violin, violas, and bassesare enabled and the cellos are disabled. The resulting mix of softsamples and loud samples for each instrument (i.e., channel) is summed,panned, and added to a master stereo output for playback. In alternativeembodiments, additional expression data can be modulated with thecross-faded signals. For example, as low frequency notes are played,haptic feedback (i.e., vibration) can be cross-faded into the compositestereo signal to simulate the rumble of bass frequencies. Similarly,other expression data (e.g., equalization data) can be cross-faded intothe output signal as would be appreciated by one of ordinary skill inthe art with the benefit of this disclosure.

Dual Fly Wheel Aspect for Modeling a Physical System Response to a UserInput

The legato gesture articulation (i.e., tap-and-swipe) provides arealistic sounding bowing action on a stringed instrument due, in part,to a virtual mass modeling system. The mass modeling system can includetwo series connected signal processors that can be analogized to avirtual dual flywheel system having mass and momentum to create smoothtransitions between notes and chords by modeling a virtual momentumcreated by a user swiping gesture on the interface 100 and maintainingthat momentum through subsequent harmonic or melodic progressions.

FIG. 11 depicts a mass modeling system (“flywheel system”) 1100,according to an embodiment of the invention. The modeling system 1100includes a first signal processing stage 1110 with a first input and afirst output, and a second signal processing stage 1120 receiving thefirst output and having a second output. The first user input caninclude the legato gesture articulation (e.g., up-and-down swipinggesture) for a virtual instrument on the interface 100. The first signalprocessing stage 1110 is coupled to the second signal processing stage1120 such that the second input of the second signal processing stage1120 receives the output of the first signal processing stage 1110. Theoutput of the second signal processing stage 1120 can determine anoutput expression level for the system 100 (e.g., a volume) in responseto the first user input. In some embodiments, the output of the flywheelsystem 1100 is configured to modulate the lifecycle of a note 1130played by a virtual instrument. The output of both the first and secondsignal processing stages 1110, 1120 can be modeled by equation (1):L=(((i*(1−h))+(L′*h))*F _(s))+C  (1)

The various components of equation 1 are defined as follows: L iscurrent level, speed, or momentum of the given flywheel, and L′ is theprevious level, speed, or momentum of the given flywheel. The externalinput ‘i’ can be a physical user input (e.g., a legato swiping motion ona touch zone) for the first signal processing stage 1110. Alternatively,the external input ‘i’ can be the output L of the first signalprocessing stage 1110 as the input to the second signal processing stage1120. The ‘h’ variable is a mass coefficient associated with eachflywheel and can vary due to a state of the second flywheel (e.g.,momentum). ‘F’ can be a scaling factor and ‘C’ can be an offsetconstant. The first term (i*(1−h)) corresponds to an influence of a newinput momentum in the system. The second term (L′*h) corresponds to theexisting momentum of the system. As can be seen in equation 1, a veryhigh mass coefficient (e.g., 0.9) will require very high input levels tomake any appreciable change to the existing momentum. Conversely, verysmall mass coefficients (e.g., 0.1) will only require small input levelsto appreciably change the existing momentum of the particular flywheel.By coupling two virtual flywheels together, the output of the secondsignal processing stage 1120 depends on a combination of the input fromthe first signal processing stage 1110 and the current momentum of thesecond signal processing stage 1120. In an alternative aspect, themomentum can be described as a rectification and low-pass filter of theinput signal (e.g., up-and-down swipe gesture over time).

An aspect of the flywheel system 1100 is to determine a resultingexpression level (e.g., volume) that produces a realistic soundingtransition from one note or chord to the next by maintaining andmodulating a virtual momentum developed in the flywheel system 1100. Toillustrate this concept, a user may initiate a very soft legatoarticulation on a touch zone by slowing swiping in an up-and-down motion(e.g., a slow first swipe speed) and gradually increasing speed until arelatively fast and constant rate is achieved (e.g., a fast second swipespeed). This can create the sound of a musical crescendo with therelative volume (i.e., expression data) of the virtual instrumentincreasing until the volume is commensurate with the momentum created bythe constant rate of the input signal. Over a period of time, bothsignal processing stages 1110, 1120 (i.e., virtual flywheels) will bevirtually rotating with the same momentum (e.g., the fast second swipespeed). As the user moves to a different touch zone (e.g., a differentchord) and begins a swipe gesture at a slower constant swipe speed(e.g., a third medium swipe speed), the flywheel system 1100 would beginplayback of the new chord with a strong volume corresponding to thesecond fast swipe speed and slowly reduce to a lower volumecorresponding to the third medium swipe speed. The rate of change can bedictated by the various terms and parameters of equation 1 that definethe virtual mass effect (i.e., momentum) of the flywheel system 1100. Itshould be noted that the flywheel system 1100 can be applied to manyapplications other than modeling a lifecycle of a note or chord. Forexample, the flywheel system 1100 could control drawing/paintingapplications where the intensity (e.g., color, width of lines,straightness of lines, line characteristics) is modulated based on theoutput of the flywheel system 1100. Alternatively, the flywheel system1100 can be configured to control a smart drum machine that alters beatparameters (e.g., beat complexity, fill complexity, sample selection,volume, etc.) based on the output of the flywheel system 1100 to providea continuously evolving drum track. It should be known that theapplication of the flywheel system 1100 is not limited to musicalapplications and may be adapted to many other systems well-suited forcontinuous modulation and control by a virtual momentum based system.

FIG. 12 illustrates an embodiment of a note lifecycle 1200, according toan embodiment of the invention. The note lifecycle can be measured in arelative level 1210 (e.g., volume) versus time 1220. The note life cycle1200 can include an attack phase 1230, a sustain phase 1240, and arelease phase 1250. In some embodiments, the note lifecycle can includeadditional phases including a decay phase or any other characteristicthat can be used to model the life cycle of a musical note. Theforegoing note lifecycle phases would be readily understood by those ofordinary skill in the art.

FIG. 13 is a simplified flow diagram illustrating aspects of a method1300 of modeling a lifecycle of a note played on a virtual instrument,according to an embodiment of the invention. The method 1300 isperformed by processing logic that may comprise hardware (e.g.,circuitry, dedicate logic, etc.), software (which as is run on a generalpurpose computing system or a dedicated machine), firmware (embeddedsoftware), or any combination thereof. The method 1300 is discussed interms of the dual flywheel physical system model 1100 (FIG. 11) asapplied to the exemplary system 700 (FIG. 7), which is configured toexecute the method 1300.

Referring to FIG. 13, the method 1300 includes receiving a touch downand motion touch gesture (e.g., a legato gesture) on a touch zone 760 ofan interface 700 (1310). At 1320, the attack phase of the note or chordbegins. In some embodiments, the level (e.g., volume) and rate of attack(ΔL/Δt) is based, in part, on a mass coefficient (h_(a)) of the dualflywheel physical system model (“flywheel system”) 1100 and the inputsignal (e.g., legato articulation). After a first predetermined periodof time (1330), the attack phase expires. The predetermined period oftime for the attack phase 1330 can be a time constant of any suitablelength as would be appreciated by one of ordinary skill in the art. At1340, the note enters the sustain phase. The length and rate of decay(ΔL/Δt) of the sustain phase is based, in part, on a mass coefficient(h_(s)) of the flywheel system 1100 and the input signal (e.g., speed ofthe legato articulation). If the input (e.g., legato gesture) stops fora second predetermined period of time (1350), the sustain phase 1240expires and the note enters the release phase 1360. The level (e.g.,volume) of the release phase decays according to a predetermined timeconstant. The length and rate of decay (ΔL/Δt) of the release phase isbased, in part, on a mass coefficient (h_(s)) of the flywheel system1100. At 1370, a lift-off condition occurs and the method ends.

In certain embodiments, the flywheel system 1100 provides a mechanism tomodulate the level and phase of a note lifecycle in real time. Asdescribed above, the sustain phase 1340 gradually decreases andeventually enters the release phase 1360 in response to a static input1350. For example, a level (e.g., output of the second virtual flywheel)of the sustain phase 1340 may decrease below a predetermined value andenter the release phase 1360. In some embodiments, the sustain phase1340 can be maintained indefinitely provided that there is a continuousinput signal 1310. Furthermore, increasing the speed of a swipe gesturecan proportionally increase a current level of the sustain phase 1340 inaccordance with the virtual momentum characteristics of the flywheelsystem 1100. In some embodiments, the note lifecycle can return (1355)from the release phase 1360 back to the sustain phase 1340 in responseto a sufficiently high input signal (e.g., swipe speed). It should benoted that the “level” described herein can refer to a volume. Inaddition, level can optionally refer to a relative “momentum” created bythe flywheel system 1100, where the different phases (e.g., attack,sustain, release) correspond to a level.

In alternative embodiments, other input signals may be used to modulatethe note lifecycle, the flywheel system 1100, or a combination of thetwo. Other user input signals can include the size of the touch pointand/or the acceleration therewith (i.e., detect the pressure or downwardforce of the input gesture), the angle of the touch signal, multipletouch signals, or any other input signal that would be appreciated byone of ordinary skill in the art with the benefit of this disclosure.

It should be appreciated that the specific steps illustrated in FIG. 13provides a particular method of modeling a lifecycle of a note played ona virtual instrument, according to an embodiment of the presentinvention. Other sequences of steps may also be performed according toalternative embodiments. In certain embodiments, the method 1300 mayperform the individual steps in a different order, at the same time, orany other sequence for a particular application. Moreover, theindividual steps illustrated in FIG. 13 may include multiple sub-stepsthat may be performed in various sequences as appropriate to theindividual step. Furthermore, additional steps may be added or removeddepending on the particular applications. One of ordinary skill in theart would recognize and appreciate many variation, modification, andalternatives of the method.

FIG. 21 is a simplified flow diagram illustrating aspects of a method2100 of modulating a lifecycle of a note, according to an embodiment ofthe invention. Method 2100 is performed by processing logic that maycomprise hardware (e.g., circuitry, dedicate logic, etc.), software(which as is run on a general purpose computing system or a dedicatedmachine), firmware (embedded software), or any combination thereof. Inone embodiment, method 2100 is performed by the processor unit 1410 ofFIG. 14.

Referring to FIG. 21, method 2100 begins with generating a userinterface on a touch sensitive display, at 2110. The touch sensitivedisplay can be similar to the user interface 100 for a virtual musicalinstrument, as shown in FIG. 1. The user interface can be implemented onany suitable touch sensitive device configured to detect touch input. At2120, method 2100 further includes generating a first virtual flywheelsystem (VFS) 1110. The first virtual flywheel system 1110 can include aninput and an output, where the input may include a user touch input onthe user interface, or any suitable data input (e.g., input signal) fromany desired source, as would be appreciated by one of ordinary skill inthe art. The output of the first virtual flywheel system can include adata value that may be an expression value, virtual momentum, or othersuitable data output that is based, at least in part, on the data inputand a mass coefficient associated with the first virtual flywheel. Themass coefficient of the VFS 1110 may be predetermined, a fixed value, adynamic value, or any suitable value as required. In some cases, theoutput of the first virtual flywheel 1110 is the product of a magnitudeof the data input and the mass coefficient.

At 2130, method 2100 further includes generating a second virtualflywheel system 1120. The second virtual flywheel system (VFS) 1110 caninclude an input and an output. In some embodiments, the second virtualflywheel system 1120 is series connected with the first virtual flywheelsystem 1100, such that the output of the first virtual flywheel system1110 is coupled to the input of the second virtual flywheel system 1120.In some cases, the input to the second virtual flywheel system 1120 mayinclude additional sources (e.g., multiple inputs). The output of thesecond VFS 1120 can be based wholly, or in part, on the input to the VFS1120 and a mass coefficient associated with the second VFS 1120. Themass coefficient of the VFS 1120 may be predetermined, a fixed value, adynamic value, or any suitable value as required. In some cases, theoutput of the first virtual flywheel 1120 is the product of a magnitudeof the data input of the second VFS 1120 and its associated masscoefficient.

At 2140, the user interface receives a user input. The user input caninclude a touch gesture, swipe gesture, or any suitable input from auser, stylus, or the like, on a touch-sensitive display. Alternatively,the input may be generated by systems other than the touch sensitivevariety. At 2150, method 2100 further includes determining a momentum ofthe first VFS 1110. As described above, the momentum may be based on theinput to the FVS 1110 and a mass coefficient associated with the VFS1110. At 2160, method 2100 further includes determining a momentum ofthe second VFS 1110. As described above, the momentum of the second VFS1120 may be based on an input signal to the second VFS 1120 and itsassociated mass coefficient. The output of the VFS 1120 can be utilizedas an expression level, an output level, a dynamic control signal, orthe like, as would be appreciated by one of ordinary skill in the art.

At 2170, method 2100 further includes applying the virtual momentum ofthe second VFS 1120 to modulate a lifecycle of a note or chord. In someembodiments, the output of the VFS 1120 (e.g., expression level) can beused to modulate one or more audio properties of an audio signal. Forexample, a volume of the audio signal can be controlled in real-timebased on a magnitude of the expression level. The output of the VFS 1120can be used to modulate any system including those not related to audioapplications. For example, the expression level may be used to controlor modulate visual properties (e.g., colors, intensity, contrast, tone)of a system. In broader contexts, the output of the VFS 1120 can be usedto modulate in real-time any desired parameter in any suitableapplication.

It should be appreciated that the specific steps illustrated in FIG. 21provides a particular method of modulating a lifecycle of a note,according to an embodiment of the present invention. Other sequences ofsteps may also be performed according to alternative embodiments. Incertain embodiments, method 2100 may perform the individual steps in adifferent order, at the same time, or any other sequence for aparticular application. Moreover, the individual steps illustrated inFIG. 21 may include multiple sub-steps that may be performed in varioussequences as appropriate to the individual step. Furthermore, additionalsteps may be added or removed depending on the particular applications.One of ordinary skill in the art would recognize and appreciate manyvariation, modification, and alternatives of the method.

FIG. 15 illustrates a computer system 1500 according to an embodiment ofthe present invention. The user interfaces described herein (e.g.,interface 100 and 700) can be implemented within a computer system suchas computer system 1500 shown here. Computer system 1500 can beimplemented as any of various computing devices, including, e.g., adesktop or laptop computer, tablet computer, smart phone, personal dataassistant (PDA), or any other type of computing device, not limited toany particular form factor. Computer system 1500 can include processingunit(s) 1505, storage subsystem 1510, input devices 1520, output devices1525, network interface 1535, and bus 1540.

Processing unit(s) 1505 can include a single processor, which can haveone or more cores, or multiple processors. In some embodiments,processing unit(s) 1505 can include a general purpose primary processoras well as one or more special purpose co-processors such as graphicsprocessors, digital signal processors, or the like. In some embodiments,some or all processing units 1505 can be implemented using customizedcircuits, such as application specific integrated circuits (ASICs) orfield programmable gate arrays (FPGAs). In some embodiments, suchintegrated circuits execute instructions that are stored on the circuititself. In other embodiments, processing unit(s) 1505 can executeinstructions stored in storage subsystem 1510.

Storage subsystem 1510 can include various memory units such as a systemmemory, a read-only memory (ROM), and a permanent storage device. TheROM can store static data and instructions that are needed by processingunit(s) 1505 and other modules of electronic device 1500. The permanentstorage device can be a read-and-write memory device. This permanentstorage device can be a non-volatile memory unit that storesinstructions and data even when computer system 1500 is powered down.Some embodiments of the invention can use a mass-storage device (such asa magnetic or optical disk or flash memory) as a permanent storagedevice. Other embodiments can use a removable storage device (e.g., afloppy disk, a flash drive) as a permanent storage device. The systemmemory can be a read-and-write memory device or a volatileread-and-write memory, such as dynamic random access memory. The systemmemory can store some or all of the instructions and data that theprocessor needs at runtime.

Storage subsystem 1510 can include any combination of computer readablestorage media including semiconductor memory chips of various types(DRAM, SRAM, SDRAM, flash memory, programmable read-only memory) and soon. Magnetic and/or optical disks can also be used. In some embodiments,storage subsystem 1510 can include removable storage media that can bereadable and/or writeable; examples of such media include compact disc(CD), read-only digital versatile disc (e.g., DVD-ROM, dual-layerDVD-ROM), read-only and recordable Blue-Ray® disks, ultra densityoptical disks, flash memory cards (e.g., SD cards, mini-SD cards,micro-SD cards, etc.), magnetic “floppy” disks, and so on. The computerreadable storage media do not include carrier waves and transitoryelectronic signals passing wirelessly or over wired connections.

In some embodiments, storage subsystem 1510 can store one or moresoftware programs to be executed by processing unit(s) 1505, such as auser interface 1515. As mentioned, “software” can refer to sequences ofinstructions that, when executed by processing unit(s) 1505 causecomputer system 1500 to perform various operations, thus defining one ormore specific machine implementations that execute and perform theoperations of the software programs. The instructions can be stored asfirmware residing in read-only memory and/or applications stored inmagnetic storage that can be read into memory for processing by aprocessor. Software can be implemented as a single program or acollection of separate programs or program modules that interact asdesired. Programs and/or data can be stored in non-volatile storage andcopied in whole or in part to volatile working memory during programexecution. From storage subsystem 1510, processing unit(s) 1505 canretrieve program instructions to execute and data to process in order toexecute various operations described herein.

A user interface can be provided by one or more user input devices 1520,display device 1525, and/or and one or more other user output devices(not shown). Input devices 1520 can include any device via which a usercan provide signals to computing system 1500; computing system 1500 caninterpret the signals as indicative of particular user requests orinformation. In various embodiments, input devices 1520 can include anyor all of a keyboard touch pad, touch screen, mouse or other pointingdevice, scroll wheel, click wheel, dial, button, switch, keypad,microphone, and so on.

Output devices 1525 can display images generated by electronic device1500. Output devices 1525 can include various image generationtechnologies, e.g., a cathode ray tube (CRT), liquid crystal display(LCD), light-emitting diode (LED) including organic light-emittingdiodes (OLED), projection system, or the like, together with supportingelectronics (e.g., digital-to-analog or analog-to-digital converters,signal processors, or the like), indicator lights, speakers, tactile“display” devices, headphone jacks, printers, and so on. Someembodiments can include a device such as a touchscreen that function asboth input and output device.

In some embodiments, output device 1525 can provide a graphical userinterface, in which visible image elements in certain areas of outputdevice 1525 are defined as active elements or control elements that theuser selects using user input devices 1520. For example, the user canmanipulate a user input device to position an on-screen cursor orpointer over the control element, then click a button to indicate theselection. Alternatively, the user can touch the control element (e.g.,with a finger or stylus) on a touchscreen device. In some embodiments,the user can speak one or more words associated with the control element(the word can be, e.g., a label on the element or a function associatedwith the element). In some embodiments, user gestures on atouch-sensitive device can be recognized and interpreted as inputcommands; these gestures can be but need not be associated with anyparticular array in output device 1525. Other user interfaces can alsobe implemented.

Network interface 1535 can provide voice and/or data communicationcapability for electronic device 1500. In some embodiments, networkinterface 1535 can include radio frequency (RF) transceiver componentsfor accessing wireless voice and/or data networks (e.g., using cellulartelephone technology, advanced data network technology such as 3G, 4G orEDGE, WiFi (IEEE 802.11 family standards, or other mobile communicationtechnologies, or any combination thereof), GPS receiver components,and/or other components. In some embodiments, network interface 1535 canprovide wired network connectivity (e.g., Ethernet) in addition to orinstead of a wireless interface. Network interface 1535 can beimplemented using a combination of hardware (e.g., antennas,modulators/demodulators, encoders/decoders, and other analog and/ordigital signal processing circuits) and software components.

Bus 1540 can include various system, peripheral, and chipset buses thatcommunicatively connect the numerous internal devices of electronicdevice 1500. For example, bus 1540 can communicatively couple processingunit(s) 1505 with storage subsystem 1510. Bus 1540 also connects toinput devices 1520 and display 1525. Bus 1540 also couples electronicdevice 1500 to a network through network interface 1535. In this manner,electronic device 1500 can be a part of a network of multiple computersystems (e.g., a local area network (LAN), a wide area network (WAN), anIntranet, or a network of networks, such as the Internet. Any or allcomponents of electronic device 1500 can be used in conjunction with theinvention.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in acomputer readable storage medium. Many of the features described in thisspecification can be implemented as processes that are specified as aset of program instructions encoded on a computer readable storagemedium. When these program instructions are executed by one or moreprocessing units, they cause the processing unit(s) to perform variousoperation indicated in the program instructions. Examples of programinstructions or computer code include machine code, such as is producedby a compiler, and files including higher-level code that are executedby a computer, an electronic component, or a microprocessor using aninterpreter.

It will be appreciated that computer system 1500 is illustrative andthat variations and modifications are possible. Computer system 1500 canhave other capabilities not specifically described here (e.g., mobilephone, global positioning system (GPS), power management, one or morecameras, various connection ports for connecting external devices oraccessories, etc.). Further, while computer system 1500 is describedwith reference to particular blocks, it is to be understood that theseblocks are defined for convenience of description and are not intendedto imply a particular physical arrangement of component parts. Further,the blocks need not correspond to physically distinct components. Blockscan be configured to perform various operations, e.g., by programming aprocessor or providing appropriate control circuitry, and various blocksmight or might not be reconfigurable depending on how the initialconfiguration is obtained. Embodiments of the present invention can berealized in a variety of apparatus including electronic devicesimplemented using any combination of circuitry and software.

While the invention has been described with respect to specificembodiments, one skilled in the art will recognize that numerousmodifications are possible: displaying the interface 100/700 and theconfiguration of the various elements therein, such as the position oftouch regions and touch zones, the types of virtual instruments used,the types of chord inversions applied to the touch zones, which may becustomizable, customizing aspects of the flywheel system, etc. Thus,although the invention has been described with respect to specificembodiments, it will be appreciated that the invention is intended tocover all modifications and equivalents within the scope of thefollowing claims.

The above disclosure provides examples and aspects relating to variousembodiments within the scope of claims, appended hereto or later addedin accordance with applicable law. However, these examples are notlimiting as to how any disclosed aspect may be implemented,

All the features disclosed in this specification (including anyaccompanying claims, abstract, and drawings) can be replaced byalternative features serving the same, equivalent or similar purpose,unless expressly stated otherwise. Thus, unless expressly statedotherwise, each feature disclosed is one example only of a genericseries of equivalent or similar features.

Any element in a claim that does not explicitly state “means for”performing a specified function, or “step for” performing a specificfunction, is not to be interpreted as a “means” or “step” clause asspecified in 35 U.S.C. §112, sixth paragraph. In particular, the use of“step of” in the claims herein is not intended to invoke the provisionsof 35 U.S.C. §112, sixth paragraph.

What is claimed is:
 1. A user interface implemented on a touch-sensitivedisplay for a virtual musical instrument comprising: a chord touchregion divided into a plurality of separate touch zones, wherein each ofthe plurality of touch zones is associated with one or more preselectedaudio signals stored in a computer-readable medium, wherein the chordtouch region is associated with a base chord, and wherein the pluralityof touch zones includes: a first touch zone having a first harmonicconfiguration of the base chord; and a second touch zone having a secondharmonic configuration of the base chord, wherein the second harmonicconfiguration includes at least one note that is higher in relativepitch than any note of the first harmonic configuration.
 2. The userinterface of claim 1 wherein the first harmonic configuration is anon-inverting base chord configuration and the second harmonicconfiguration is a first inversion configuration.
 3. The user interfaceof claim 1 wherein the plurality of touch zones further includes a thirdtouch zone, wherein the third touch zone includes a third harmonicconfiguration of the base chord that includes at least one note that ishigher in relative pitch than any note of the second harmonicconfiguration.
 4. The user interface of claim 3 wherein the firstharmonic configuration is in a base chord configuration, the secondharmonic configuration is a first inversion of the base chord, and thethird harmonic configuration is a second inversion of the base chord. 5.A computer-implemented method comprising: displaying, by a processor, avirtual musical instrument on a touch-sensitive display, the virtualmusical instrument including a chord touch region divided into aplurality of separate touch zones, wherein each of the plurality oftouch zones is associated with one or more preselected audio signalsstored in a computer-readable medium; wherein the chord touch region isassociated with a base chord, and wherein the plurality of touch zonesincludes: a first touch zone having a first harmonic configuration ofthe base chord; and a second touch zone having a second harmonicconfiguration that is higher in relative pitch than the first touchzone; receiving, by the processor, a user input corresponding to touchor swipe on one of the first or second touch zones; and playing thecorresponding harmonic configuration of the base chord based on the userinput.
 6. The computer-implemented method of claim 5 wherein the firstharmonic configuration is a non-inverting base chord configuration andthe second harmonic configuration is a first inversion configuration. 7.The computer-implemented method of claim 5 wherein the plurality oftouch zones further includes a third touch zone, wherein the third touchzone includes a third harmonic configuration of the base chord that ishigher in relative pitch than the second harmonic configuration.
 8. Thecomputer-implemented method of claim 7 wherein the first harmonicconfiguration is in a base chord configuration, the second harmonicconfiguration is a first inversion of the base chord, and the thirdharmonic configuration is a second inversion of the base chord.
 9. Acomputer-implemented system comprising: one or more data processors; oneor more non-transitory computer-readable storage media containinginstructions configured to cause the one or more processors to performoperations including: displaying a virtual musical instrument on atouch-sensitive display, the virtual musical instrument including achord touch region divided into a plurality of separate touch zones,wherein each of the plurality of touch zones is associated with one ormore preselected audio signals stored in a computer-readable medium;wherein the chord touch region is associated with a base chord, andwherein the plurality of touch zones includes: a first touch zone havinga first harmonic configuration of the base chord; and a second touchzone having a second harmonic configuration that is higher in relativepitch than the first touch zone; receiving a user input corresponding totouch or swipe on one of the first or second touch zones; and playingthe corresponding harmonic configuration of the base chord based on theuser input.
 10. The system of claim 9 wherein the first harmonicconfiguration is a non-inverting base chord configuration and the secondharmonic configuration is a first inversion.
 11. The system of claim 9wherein the plurality of touch zones further includes a third touchzone, wherein the third touch zone includes a third harmonicconfiguration of the base chord that is higher in relative pitch thanthe second harmonic configuration.
 12. The system of claim 11 whereinthe first harmonic configuration is in a base chord configuration, thesecond harmonic configuration is a first inversion of the base chord,and the third harmonic configuration is a second inversion of the basechord.